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It  is  the  mission  of  the  Operational  Test  and 
Evaluation  Force  (OPTEVFOR)  to  ensure 
that  all  systems  are  as  effective,  suitable,  and 
survivahle  as  possible  and  convey  their 
capabilities  and  limitations  to  the  warfighter. 
In  this  regard,  the  test  and  evaluation  (T&E) 
community  has  the  unique  opportunity  to  provide 
advice,  guidance,  and  the  wisdom  gained  from  our 
exposure  to  the  TScE  of  hundreds  of  systems  providing 
warfighting  capability  over  the  past  60  years  and  the 
lessons  learned  from  their  successes  and  failures. 

About  OPTEVFOR 

The  U.S.  Navy’s  OPTEVFOR,  headquartered  in 
Norfolk,  Virginia,  is  the  Navy’s  sole  independent 
Operational  TScE  command.  It  is  our  mission  to 
report  to  the  Chief  of  Naval  Operations  (CNO)  on  the 
operational  effectiveness  and  operational  suitability  of 
new  and  improved  warfighting  capability. 

In  effect,  we’re  information  providers.  However, 
we  do  not  limit  our  service  to  the  final  examination 
of  a  warfighting  capability.  We  are  active  participants 
in  all  phases  of  product  development  providing 
operational  insight  and  identifying  both  real  and 
potential  operational  shortfalls  to  a  system’s  end- 


OPTEVFOR 
Who  we  are: 

•  Navy’s  Independent  OT&E  Agent 

•  Reporting  Directly  to  CNO 

•  Information  Providers  to  Decision  Makers 
and  Warfighters 

•  Supporting  both  Acquisition  and  Experi¬ 
mentation 


state  capability.  As  Dr.  McQueary,  (Director, 
Operational  Test  and  Evaluation,  Office  of  the 
Secretary  of  Defense)  has  said,  “OT&E  should  be  a 
period  of  confirmation,  not  discovery”  and  we  can 
only  achieve  that  by  being  there  throughout  the 
process. 

How  do  we  do  that  and  with  what  resources? 
Figure  i  is  a  high  level  view  of  OITEVFOR’s 
portfolio.  This  figure  provides  a  view  of  how  we  are 
organizationally  aligned,  both  internally  and  externally, 
and  our  resources.  Bottom  line  is,  fewer  than  300 
people,  an  operating  budget  of  just  over  111  million 
annually,  and  an  average  per  year  of  140  million  of 
reimbursable  funding  from  program  managers. 

OPTEVFOR  is  currently  assigned  OT&E  respon¬ 
sibility  for  over  300  programs.  These  programs  span 
the  Navy’s  warfighting  enterprises  of  aviation,  surface, 
subsurface,  command  and  control,  and  expeditionary 
warfare.  The  wide  range  of  product  types  that  support 
new  and  improved  warfighting  capability  requires  a 
diverse  level  of  operational  and  technical  experience 
within  the  staff  at  OPTEVFOR  in  order  to 
adequately  support  our  mission.  This  diversity  is 
obtained  by  the  continuing  rotation,  average  3  year 
tour,  of  fleet  operational  personnel  that  serve  as  the 
operational  test  coordinators  and  operational  test 
directors. 

OPTEVFOR  initiatives 

The  Navy  T&E  community  in  general  and  OP¬ 
TEVFOR  specifically  are  actively  engaged  in  develop¬ 
ing  methods,  procedures,  and  policies  to  help  ensure 
that  the  financial  investments  of  the  U.S.  Department 
of  Defense  (DoD)  and  the  Department  of  the  Navy  are 
provided  the  opportunity  to  generate  the  highest  rate 
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Figure  1.  Operational  Test  and  Evaiuation  Force  portfolio 


of  return  possible  in  warfighting  capability  and 
preservation  of  human  life.  Four  specific  initiatives 
currently  ongoing  are: 

1.  The  Navy’s  enterprise  approach  to  T8cE  (TScE 
Board  of  Directors) 

2.  T8cE  cycle  time  reduction 

3.  Mission-based  test  design 

4.  Integrated  testing 

Navy’s  enterprise  approach  to  T&E 

As  the  result  of  a  recommendation  to  the  chief  of 
Naval  Operations  (CNO)  in  July  2005  that  stated: 
“The  need  for  a  T&E  capability  that  is 
synchronized  with  product  procurement 
across  the  spectrum  is  critical  to  support  the 
Navy’s  future  war  fighting  needs.  As  part  of 
the  enterprise  integration  of  T&E  there  is  a 
need  for  a  single  Navy  T&E  process  owner. 

An  additional  dimension  is  the  need  for  a 
more  effective  and  efficient  business  model  in 
the  relationship  between  government  and 
industry,  the  desire  to  achieve  synergism, 


and  produce  a  “win-win”  situation  for  both 
entities.” 

As  a  result  of  this  recommendation,  the  Navy’s  Test 
and  Evaluation  Board  of  Directors  was  formally 
established  by  the  Assistant  Secretary  of  the  Navy  for 
Research,  Development,  and  Acquisition  in  April 
2007.  This  board  of  directors  is  composed  of  Flag 
and  Senior  Executive  stakeholders  from  the  CNO 
staff.  Navy  System  Commands,  Program  Executive 
Offices,  and  the  Marine  Corps,  and  is  co-chaired  by 
the  Department  of  the  Navy  T&E  Executive  and 
COMOPTEVTOR.  It  is  the  function  of  the  Board  of 
Directors  to  apprise  department  leadership  of  T&E 
enablers  to  ensure  that  the  needs  of  acquisition 
programs  are  met  and  balanced  with  overall  Navy 
warfare  enterprise  goals.  The  Board  resolves  issues 
among  T&E  enablers,  recommends  priorities  to  be 
executed  by  the  responsible  organizations,  supports  the 
Navy  enterprise  and  integrated  T&E  process,  and 
champions  improvement  initiatives  to  meet  program 
requirements  while  continually  improving  T&E  cost 
efficiency. 
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Figure  2.  Navy  test  and  evaluation  board  of  directors 


To  this  end,  the  vice  chief  of  Naval  Operations 
(VCNO),  the  assistant  commandant  of  the  Marine 
Corps,  and  the  assistant  secretary  of  the  Navy  for 
Research  Development  and  Acquisition  realigned  the 
reporting  requirements  of  the  Navy  T8cE  executive  to 
not  only  the  VCNO  hut  also  to  the  assistant  secretary 
for  Research  Development  and  Acquisition.  What  this 
means  is  now  the  Department  of  the  Navy,  Navy,  and 
Marine  Corps,  have  a  single  process  owner  for  TScE. 
And,  that  process  owner  reports  to  and  represents  not 
just  the  CNO  hut  also  the  Navy’s  service  acquisition 
executive  on  aU  T&E  policy.  This  is  a  very  powerful 
move  because  now  there  is  a  chain  of  command 
between  the  CNO,  the  person  ultimately  responsible 
by  law  for  training  and  equipping  the  fleet,  and  the 
business  decision  maker  on  the  acquisition  side  that 
provides  the  CNO  with  warfighting  products. 

T&E  cycle  time  reduction 

Over  the  past  several  years  the  decision  cycle  by  the 
development  and  acquisition  community  to  provide 
products  to  the  fleet  has,  in  many  instances,  been 
reduced  beyond  the  ability  of  the  operational  tester  to 
provide,  in  a  timely  manner,  value-added  information 
to  the  decision  maker.  Reduced  cycle  time  of  product 
development  and  fielding  for  highly  software  intensive 
systems  now  ranges  from  several  months  to  less  than  a 
year.  In  addition,  the  final  capability  that  is  delivered  to 
the  fleet  is  more  often  than  not,  not  determined  until 


the  platform  is  ready  for  deployment.  And  let’s  never 
forget  it  is  all  about  the  warfighters  needs!  The  ability  of 
the  operational  tester  to  plan,  based  on  proposed  yet 
uncertain  final  system  capability  and  configuration, 
schedule  warfighter  resources,  execute  the  test,  conduct 
the  analysis  and  write  an  evaluation  report  is  severely 
hampered  by  the  time  it  takes  to  conduct  these 
activities  in  the  long  established  methods  and  processes 
currently  utilized  {Figure  2). 

Operational  testing  has,  over  the  past  10  years  taken 
on  a  more  technical  analysis  of  the  capabilities  provided 
by  developers.  The  ability  to  collect  system  perfor¬ 
mance  data  at  levels  significantly  below  the  operational 
level  (that  is  the  operator  and  supervisor  level)  has 
resulted  in  a  belief  that  it  is  the  responsibility  of  the 
operational  tester  to  conduct  extensive  “failure  analysis” 
on  system  performance  when  a  shortcoming  or  failing 
is  identified  during  test.  This  belief  is  counter  to 
evaluating  a  system’s  performance  capability  at  the 
operational  mission  level.  By  elevating  the  level  of 
evaluation  back  to  the  operational  mission  level,  it  is 
possible  to  reduce  the  evaluation  and  report  time  on  a 
warfighting  capability.  This  elevating  of  the  evaluation 
level,  combined  with  leveraging  all  available  data  and 
information,  wherever  it  is  created,  contractor,  pro¬ 
gram  office,  or  operational  test  wiU  allow  for 
significantly  greater  insight  in  a  systems  performance 
capability  as  well  as  allow  for  greater  confidence  in  that 
system  after  it  is  deployed. 
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Figure  3.  Integrated  test 


Mission-based  test  design  (MBTD) 

MBTD  is  a  detailed  and  disciplined  systems 
engineering  approach  to  test  design.  Test  design  is 
done  early  in  the  acquisition  life  cycle  (prior  to  MS  B  if 
possible).  Simply  put,  the  methods  and  processes  we 
have  developed  utilize  existing  measures,  the  Universal 
Joint  Tasks  Lists  (UJTS)  and  the  Navy  Mission 
Essential  Task  Lists  (NMETLS)  to  collect  data  on 
the  system  and  the  operators’  capability  to  execute  the 
mission.  The  foundation  document  produced  from  this 
process,  known  as  the  OT  framework,  provides  the 
groundwork  for  all  subsequent  detailed  test  plans.  In 
fact,  the  OT  framework  contains  enough  detail  itself  to 
serve  as  a  test  plan.  Test  size  is  based  on  the  conditions 
which  impact  the  system  and  warfighter  in  their 
environment.  Statistical  methods  known  as  Factorial 
Designs  and  Design  of  Experiment  produce  permuta¬ 
tions  of  the  tasks  based  upon  the  number  of  conditions. 

Tasks  are  broken  into  logical  executable  segments 
called  vignettes.  Vignettes  can  be  as  small  as  a 
maintenance  demonstration  or  as  large  as  an  end-to- 
end  exercise.  The  end  product  out  of  the  MBTD,  the 
OT  framework,  contains  a  detailed  description  of  test 
vignettes,  conditions  in  which  the  test  is  to  be 
conducted,  the  measures  of  success  and  a  robust, 
detailed  list  of  resources  needed  to  execute.  The  resource 
list  can  then  be  fed  into  the  TScE  Master  Plan. 


Integrated  testing 

The  May  2003  version  of  the  DoD  Directive  on 
Defense  Systems  Acquisition  specifically  states  “Test 
and  evaluation  shall  be  integrated  throughout  the 
defense  acquisition  process.”  What  has  been  lacking 
until  recently  is  the  answer  to  the  question  “What  does 
that  mean?”  Eor  the  Navy,  that  means,  and  I  quote  from 
the  Navy’s  T8cE  Board  of  Directors,  “Integrated  testing 
is  the  collaborative  planning  and  collaborative  execution 
of  test  phases  and  events  to  provide  data  in  support  of 
independent  analysis,  evaluation,  and  reporting  by  all 
stakeholders  particularly  the  developmental  (both  con¬ 
tractor  when  appropriate  and  government)  and  opera¬ 
tional  test  communities.”  (See  Figure  3)  A  poUcy  memo 
is  expected  out  soon  on  this  subject. 

We  are  also  working  with  DOT8cE  and  USD 
(AT&L)  in  revising  the  format  for  T&E  Master  Plans 
to  reflect  a  more  integrated  approach. 

How  do  we  proceed? 

All  of  this  raises  some  very  challenging  questions. 
What  is  the  impact  of  these  initiatives  on  the  industry? 
How  does  it  affect  the  acquisition  community  and  the 
total  test  community?  How  will  it  affect  the  govern¬ 
ment/industry  relationship?  Are  there  statutory  or 
regulatory  barriers  to  succeeding  in  these  initiatives? 
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All  good  and  valid  questions,  do  we  have  the  answers? 
No.  Can  we  work  together  to  come  up  with  the 
answers?  That’s  the  whole  point,  collaboration.  Mean¬ 
ingful  and  open  dialogue  and  action  about  what  we  all 
know  needs  to  be  done  but  have  avoided  because  it’s 
believed  too  difficult.  It  is  difficult,  but  not  impossible. 
Government  and  industry  have  different  agendas. 
Admit  it,  understand  it,  and  move  on.  We  once  again 
are  in  a  time  when  we  are  faced  with  significant 
challenges.  This  time  let’s  look  at  these  challenges  as 
opportunities  to  move  beyond  our  parochial  stove 
piped  interests  and  really  do  what  we  say:  Support  the 
warfighter.  □ 

Rear  Admiral  (RDML)  Steve  Voetsch  was  born  in 
Fort  Lewis,  Washington.  He  received  his  commission  in 
1979  from  the  United  States  Naval  Academy  and  was 
designated  a  naval  flight  officer  in  1 980. 

After  initial  F-4  Phantom  training  in  VF-1 71  he  was 
assigned  to  VF-74  where  he  completed  his  first  deployment 
to  the  Indian  Ocean  on  board  USS  Forrestal  ( CV 59),  in  the 
last  East  Coast  F-4  Phantom  squadron.  In  1983  he 
transitioned  to  the  F-14  Tomcat  and  completed  a  follow-on 
deployment  on  board  USS  Saratoga  (CV  60)  with  VF-74, 
deploying  to  the  Mediterranean  Sea. 

After  a  brief  shore  tour,  he  was  assigned  to  VF-143 
embarked  in  USS  Dwight  D.  Eisenhower  ( CVN  69).  From 
1988—1991,  RDML  Voetsch  served  as  the  operations  and 
maintenance  officer  in  VF-41,  operating  from  USS 
Theodore  Roosevelt  (CVN  71),  flying  numerous  combat 
missions  over  Iraq  and  Kuwait  during  Operations  Desert 
Shield/Storm  and  Provide  Comfort.  Assuming  command  of 
the  “Fighting  Diamondbacks”  of  VF-102  in  September 
1 995,  RDML  Voetsch  was  directly  responsible  for  leading 
the  squadron  toward  earning  the  Battle  “E”,  Safety  “S”,  the 
coveted  Clifton  Award,  Grand  Slam  Award,  Tactical  Air 
Reconnaissance  POD  System  (TARPS)  Award,  and  the 
NAS  Oceana  Athletic  Award.  RDML  Voetsch  commanded 
the  “Grim  Reapers”  of  VF-101  from  July  1998  to  December 
1999.  In  July  2000  he  reported  as  deputy  commander. 
Carrier  Air  Wing  One  and  assumed  duties  as  commander. 
Carrier  Air  Wing  One  in  July  2001. 

Serving  ashore,  RDML  Voetsch  was  assigned  as  a  fighter 
instructor  in  VF-101.  Other  shore  assignments  include  the 
Armed  Forces  Staff  College  and  assistant  Washington 
placement  officer  in  the  Bureau  of  Naval  Personnel;  aide/ 
flag  lieutenant  to  the  chief  of  naval  personnel.  Admiral 
Ronald  J.  Elatoper;  two  tours  as  executive  assistant  to  the 
commander.  North  American  Aerospace  Defense  Command 
and  United  States  Northern  Command.  RDML  Voetsch 
served  as  the  deputy  chief  of  staff  for  operations,  training 
and  readiness  (N3/N7)  on  the  staff  of  commander,  US. 


Pacific  Fleet  from  July  2005  to  May  2007.  On  May  24, 
2007,  RDML  Voetsch  assumed  command  of  Operational 
Test  and  Evaluation  Force  in  Norfolk,  Virginia. 

Steven  K.  Whitehead  currently  serves  as  the  executive 
director  to  the  Commander,  Operational  Test  and  Evalu¬ 
ation  Force  ( COMOPTEVFOR),  US.  Department  of  the 
Navy.  He  was  selected  to  a  senior  level  (SL)  position  as 
a  senior  executive  on  June  1 7  2001.  He  enlisted  in  the  US. 
Navy  out  of  high  school  in  1975  as  a  suface  electronic 
wafare  technician  and  continued  to  serve  the  Navy  as 
a  civilian  while  attending  the  University  of  Rhode  Island, 
earning  an  undergraduate  degree  in  electrical  engineering. 
He  transferred  to  the  Naval  Wafare  Assessment  Center 
(NWAC),  Corona,  California  where  he  served  in  a  variety 
of  engineering  and  weapon  systems  analysis  positions  from 
1985  to  1991.  His  experience  includes  weapon  system  and 
missile  flight  analysis  in  the  Suface  Weapons  Department, 
Flight  Analysis  Division,  Point  Defense  Systems  Branch, 
peforming  engineering  evaluations  of  Point  Defense 
(RIM-7E,  mH  &‘M  and  RAM)  suface  missile  firings 
and  weapon  system  exercises;  project  leader  in  the  Systems 
Analysis  Division,  Special  Projects  Branch  for  the  Vertical 
Launch  ASROC  weapon  System,  the  Mk  116  Mod 
0  Vertical  Launching  System,  and  the  TRIDENT  II 
(D5)  Strategic  Weapon  System  (SWS).  The  latter  two 
projects  directly  supported DEPCOMOPTEVFOR,  Pacific 
and  COMOPTEVFOR,  respectively.  For  the  TOMA¬ 
HAWK  SEARA  Branch,  he  continued  his  duties  as  NWAC 
project  lead  in  support  of  OPTEVFOR  for  the  TRIDENT 
II  (D5)  SWS,  and  was  assigned  the  duties  of  TOMA¬ 
HAWK  quality  assurance  service  test  representative  to  the 
Cruise  Missile  Program  Office,  San  Diego,  California.  He 
also  served  on  the  department  staff.  Surface  Weapons 
Department,  coordinating  the  establishment  of  an  antisub¬ 
marine  wafare  division. 

In  May  1991,  he  transferred  to  OPTEVFOR  Head¬ 
quarters,  No  folk,  Virginia,  as  an  operations  research 
analyst.  He  was  selected  as  the  deputy  assistant  chief  of 
staff  for  C4I  Systems  in  October  of  the  same  year  and  held 
that  position  until  June  1 996  at  which  time  he  was  selected 
to  assume  the  duties  as  the  technical  director.  In  November 
2007,  his  duties  were  realigned  to  better  support  the 
commander  and  his  position  was  designated  as  the  executive 
director. 

Steven  Whitehead  received  his  bachelors  degree  in 
electrical  engineering  from  the  University  of  Rhode  Island. 
He  also  holds  a  masters  degree  in  management  from  Troy 
University  and  is  a  1994  graduate  of  the  Naval  War 
College.  His  awards  include  the  Navy  Civilian  Superior 
Service  Medal,  Navy  Civilian  Meritorious  Service  Medal, 
two  Sustained  Superior  Peformance  awards.  Special 
Mission  Support  award,  and  three  Letters  of  Commenda¬ 
tion. 
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